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The Operation with Economic Efficiency (OEE) Program in the Republic of 
Korea Air Force (ROKAF) was designed to reduce the cost and improve the efficiency 
of all ROKAF operations. The ROKAF is now faced with the problem of developing 
a more efficient pilot candidate selection and training program which will improve pilot 
quality and reduce attrition rates and the number of aircraft accidents. The present 
research provides a starting point for the development of a more efficient pilot 
selection and training program. If successful this program should result in the selection 
of pilot candidates exhibiting a significantly higher probability of completing the pilot 
training program. This thesis also provides a summary of aviation selection test 
batteries presently in use by the U.S. Air Force and U.S. Navy and applies the 
principles of decision theory in determining whether a change in the ROKAF'’s present 
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[. INTRODUCTION 


A. BACKGROUND 

Throughout the history of aviation there has been constant concern with 
selecting those individuals with both mental and physical qualities desirable for 
optimum performance in flight environments. The intensity and quantity of research 
performed over the years in the particular area of pilot selection highlights the fact that 
the key to combat air superiority is the selection of high quality pilot candidates. 
During World War II and the Korean conflict it became further apparent that the 
ability to gain total tactical and strategic advantage over the enemy depended on 
whether or not air superiority was achieved. Fortunately the U.S. was able to achieve 
this superiority in the air during the last three major conflicts. Unfortunately there is a 
growing consensus among military analysts that the U.S. may be lulling itself into a 
sense of complacency. The following quotation from a publication by B. K. Holloway 
underscores the basis for this growing complacency: [Ref. 1] 


e “Twenty-six million living Americans are veterans of military service, and most 
of them have served in wartime. How many of these 26 million ever had to face 
an enemy who held air superiority ? ... In all, probably no more than one out 
of 150, for after Februarv 1943 the U.S. and allies had undisputed air 
superiority in the Mediterranean and Western Europe; after August of 1943 we 
had it in the Pacific... In Korea we won air superiority twice - from the North 
Korean Air Force in the first two months of that war, and again from the 
Chinese Air Force after November 1950. In South Vietnam, our air superiority 
has come by default. In North Vietnam it has yet to be seriously challenged.” 


Today we have reached an age of strategic parity where Eastern block 
technological developments in air combat systems are advancing at a rate comparable 
with those as the West. The key in future conflicts thus becomes improved combat 
effectiveness and thus, ultimately, the development of improved aircrew selection and 
training programs. 

The essence of these programs might be summarized simply as: 

1) Using an effective selection method, find the best candidate pilot, 
2) Provide him/her with the best and most cost effective training, 


3) Significantly improve retention rates to maximize the training investment. 


The first step, selecting the best pilot candidate, is not an easy one. The key 
questions become, “What mental and physical characteristics do we look for ?” and 
“What are our selection criteria ?” The answers to these two basic questions have been 
the object of much of the pilot candidate selection research, which will be addressed in 
the following section. 

The ROKAF 1s currently experiencing an excessive and unacceptable attrition 
rate in its pilot training programs. This problem is further compounded by the fact 
that the total number of candidates admitted to the flight training program has steadily 
increased since the early 1980's, while training resources remained limited. As a result 
the candidates who remain in the program are afforded fewer hours of instruction in 
the air and thus do not receive the desired quality or quantity of training. 

In the early 1960’s both the U.S. Navy and Air Force flight training programs 
suffered similar attrition problems. Figure 1.1 provides a summary of the attrition 
categories for the respective services at that time and Appendix A provides a detailed 
description of attrition categories. The comparison of attrition rates between USN and 
USAF presented in Figure 1.1 as a result of various factors. USN had 60.5% attrition 
rate and 21.5% of that was due to flight failure. Similarly USAF had 49.0% due to 
flight failure out of 33.8% attrition. However, with the significant improvement in 
existing selection methods and the incorporation of new methods of psychological 
screening both services were able to decrease their respective attrition rates by almost 
30% by 1977 )[Retee|: 

The present research attempts to summarize some of these improved selection 
methods utilized by both the U.S. Navy and Air Force and presents recommendations 
based on this summary for improving the pilot selection program of the ROKAF. 


B. OBJECTIVE 

The Korean Air Force (ROKAF) needs a more effective way to select and predict 
performance of pilot candidates for success in primary flight training and operational 
flight training to maintain its air superiority. The purpose of this thesis, therefore, 1s to 
suggest a pilot selection system which selects potential successes in flight training and 
improves pilot quality. This research also provides a) an overview of pilot selection 
research, b) testing methods, and c) a related decision theory and cost effectiveness 
model. 
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Representative Navy and Air Force Pilot Attrition Categories (From Ref.2). 


II. U.S. PILOT SELECTION RESEARCH 


A. WORLD WARS 

It was first noted during WW I that a significant number of aircraft accidents 
were due not primarily to aircraft failures but to human error. Thus, based on this 
data France, Italy and England all initiated programs to aid in the prediction of pilot 
success during primary flight training. U.S. efforts began as early as 1919. (Ref. 3] 
Early selection tests in U.S. were based prmarily on _  Paper-and-Pencil 
Perceptual/Cognitive tests with psychomotor devices and were able to predict flight 
performance of pilot candidates with resonable validity. 

The development of selection tests continued between the First and Second 
World Wars. Much of this effort was aimed at the development of psychomotor test 
devices which simulated stick and rudder movements of the aircraft. 
Perceptual/Cognitive Paper-and-Pencil tests likewise continued to be developed but 
both these and the psychomotor tests lacked significant predictive validity. A 
contributor to this fact was the deficiency in external criteria of in-flight performance. 
During WW II the demand to train large numbers of pilots in a short amount of time 
along with increased training cost created the necessity for minimizing attrition due to 
poor proficiency. The Navy thus incorporated in its selection test battery numerous 
items previously evaluated in the civilian pilot training program. In the “Pensacola 
1000 aviator study”, evaluation was made of the validity of more than 60 psychological, 
physiological and psychomotor test items from the Navy selection test battery utilizing 
over 900 Navy flight candidates. Results indicated that psychological and psychomotor 
measures exhibited greater validity for the prediction of success in flight training than 
did physiological measures and verified the validity and effectiveness of these 
psychological instruments and psychomotor tests in the prediction of flight success. 
(Ref. 3] 

The US Army and Air Force also developed their pilot selection testing battery 
during WW II and several useful methods such as the Complex Coordination test and 
Divided Attention test are still used in the present selection test battery. The Complex 
Coordination test, similar to USN one, used by the requires the candidate to make 


simple controlled movements of a stick and rudder in response to patterns of visual 
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stimuli. During WW II it was the most useful and.most highly weighted test. Other 
testing methods developed included the Finger Dexterity, Rudder Control Test and 
Two Hand Pursuit Test. Typical apparatus test validity coefficients for pilots, 
navigators and bombardiers are presented in Table 1 [Ref. 4]. Note that the test most 
highly related to: pilots is the Complex Coordination Test. The Discrimination 


Reaction Time 1s significantly related to navigators and bombardiers. 


TABLE I 
VALIDITY OF APPARATUS TESTS 


(year, sample size) 


Complex Coordination Pilots Navigators Bombardiers 


(1933 ay (ee 1022) La eae 
33, Ee 40 me y=: 24 10, an 
Rotarv Pursuit with Au 1943, i) 

Divided Attention 14, ¢r ia = .22 


Finger Dexterity (1943, a) (1943, ay) (1943, noe 
r=.07, = 0 r=. LOT = 3° r= 71 io 5 


Discrimination ee eg Eye 1022 (1942, 1829 
Reaction Time Miee OO: sha. War 


Rudder Control Test WS pee 


Two Hand Pursuit (1943, 1385) 
r=.27, (average) 


Two Hand ee os 5g r= 99 ee Dee 


Coordination ae 


r = validity coefficient based on dichotomous pass/fail training criterion. 


rt, = validity coefficient corrected for restriction in range. 


(From Reference 4) 
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B. POST WAR RESEARCH ; 

Psychomotor Tests were used in USAF selection battery until 1951 but were 
deleted due to administrative, quality control and reliability problems. A subsequent 
study of the revision of the psychomotor tests indicated that the use of simple motor 
skill tests offered little if any predictive value for flight school success. It was 
concluded that this was due to: [Ref. 5] 

1) The unreliability of both predictor and criterion scores, 


2) The use of inappropriate motor tests for the particular criteria being 
evaluated, and 


3) The task of flying an airplane is too complex for simple motor skill tests to be 
of substantial benefit. 


This indicated that other more complex abilities were necessary for successful 

pilot performance and that the goal of future research should be to: 
1) Define the abilities, 
2) Quantify the basic behaviors making up the abilities, 
3) Devise accurate test measures. 

These problems were considered continuously through the 1970s. As a result, 
currently the USAF has been achieving great success in predicting flight performance 
in its UPT program. This success will be addressed in more detail in Chapter III. 

The other type test which has been the most widely and easily used as a selection 
test has been the Perceptual/Cognitive Paper-and-Pencil tests. The U.S. military 
service paper-and-pencil test battery consisted of: [Ref. 3] 

1) A general intelligence component composed of verbal and quantitative items, 
2) | Mechanical comprehension, 
3) A spatial component, and 


4) A background or biographical inventory composed of miscellaneous subset of 
items usually of an historical nature known to relate to aviator success. 


The analysis of EEG (Electroencephalogram) recordings has also received much 
attention as a possible predictor of aircraft accidents. It has been found that 
individuals with abnormal EEG’s are involved in accidents at three times the rate of 
those individuals with normal EEG’s! [Ref. 3}. As a result of these and similar findings 
the Navy now screens all new student aviators for abnormal EEG’s prior to 


commencement of flight training. 


‘These findings have resulted in the Danish Air Force’s use of EEG recordings in 
the selection of pilot candidates. 
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C. CURRENT RESEARCH 

The establishment of reliable criterion measures is a prime factor in any selection 
process. This continues to be an extremely difficult and dynamic task. For example, 
the Navy has recently found that selection variables or criterion used to predict 
undergraduate training success and early aviation training performance measures may 
not be related or may even be inversely related to performance in the advanced 
operational environment [Ref. 3]. 

One facility which has been invaluable in providing insight into the establishment 
of viable performance criterion for possible use in selection tests is the Navy Air 
Combat Maneuvering Range (ACMR). This facility allows multiple fighter aircraft to 
engage in a tactical environment, allowing the simulated employment of air to air 
Weapons as a means of providing training in tactical skills, weapon utilization and 
weapon envelope recognition. Several problems arise from utilizing ACMR generated 
criterion, however. First, the high psychological stress levels associated with actual 
combat may be absent, secondly, individual performance is dependent on adversary 
performance. Thus an independent performance assessment is difficult if not 
impossible. Plans are ongoing to better utilize this facility in understanding aviator 
skills which appear to enhance ACMR performance. 

Finally, Perceptual Psychomotor Testing and Division of Attention Tests are 
being improved by both the U.S. and Israeli Air Force respectively. Psychomotor 
testing has already made a unique contribution to the USAF UPT selection program. 
Israeli Air Force has likewise experienced success in predicting training success in high 
performance jet aircraft using the Division of Attention Test? [Ref. 6]. 

We have now discussed what testing methods have been developed and exist 
today, and have established how important they are in developing the pilot selection 
program. We now suggest useful pilot selection methods for the ROKAF which 
hopefully will result in considerable resource savings and significant improvements in 


pilot quality. 


Approximately for 2,000 subjects, the data was significantly related to success in 
jet training. 
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Il. AVIATOR SELECTION METHODS FOR ROKAF 


Although the human resources and materials for the flight training program in 
the ROKAF have increased only slightly, the number of pilot candidates is increasing 
every year as the number of Korean Air Force Academy (KAFA) cadets increases. 
Since the only admission requirement to the KAFA for flight training is a medical 
clearance, and most pass the medical examination, the ROKAF pilot training 
command cannot handle the number of cadets and candidates from other sources 
(AFROTC and Navy). This increased number of pilot candidates has created the 
necessity for minimizing the number of candidates lost due to poor or unsatisfactory 
proficiency in the UPT program; maximizing the probability that potential failures will 
be screened out prior to flight training. 

The present pilot selection procedures of the ROKAF do not adequately take 
into account candidates’ mental abilities. The electrocadiogram, however, is being used 
for fighter pilots, and an abnormal electrocadiogram will prohibit continuation of flight 
training. All of the KAFA cadets who pass the medical examination enter the flight 
training program and have a chance to fly. Flight training starts early before 
graduation from the KAFA and ends about a year after graduation. This system, with 
flight training begining early before graduation was adapted during the 1980's. 
However the current system does not appear to represent a significant improvement in 
previous program. Cadets who fail as a cadet find it difficult to continue the program 
and those who fail as an officer after graduation and commissioning find it difficult to 
adapt to different specialties. Candidates are divided into several subgroups in order to 
meet the limited training squadron resources. Graduates of the subgroups enter the 
next phase of training with attrites returning to the KAFA. Attritions continue 
through the flight training - from the primary training program up to the advanced 
program over several years. The only way that candidates could be differentiated is on 
some amount of actual flying at the training squadron and the failures would be totally 
subjective based on instructors’ feeling, knowledge and their experiences, except for 
voluntary withdrawal or drop on request (DOR). 

The following demonstrates a useful way to help in solving the existing short 
comings of the current ROKAF pilot training program and suggests pilot candidate 


selection methods which might be used as a starting point in alleviating these problems. 
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A. METHODS P. 

This section suggests selection methods for the ROKAF which are currently 
being used in the U.S. The purpose of the tests are to select candidates who have a 
high probability of completing the flight training program. Multitask performance 1s 
being used in USN and it has enjoyed considerable success in predicting flight training 
[Ref. 7]. The USAF uses a psychomotor screening system for selecting candidates as 
well as identifying whether the candidate is suited for fighter pilot training [Ref. 8]. 
The USAF also suggests that an integrated model including psychomotor testing and 
other available information - AFOQT scores, technical degree, prior flying experience, 
age of subjects, etc. - is better than using only the psychomotor test. All three 
methods use a psychomotor test since the psychomotor scores have been found to 
relate significantly to flight training pass/fail criteria. As the USN adapted the EEG 
recordings as a direct result of the Danish Air Force as we discussed in Chapter II, the 
ROKAF may choose one of these three methods or a combination and apply it to 
ROKAF cadet selection with some possible modifications. The following text describes 
the three methods which could be used and modified for the ROKAF and shows how 
the predicted results may be related to the actual outcomes. 

1. Multitask Performance 

The USN had great success in predicting student success in primary flight 
training utilizing multitask performance tests in 1986 [Ref. 7]. A Psychomotor Test 
(PMT) and a Dichotic Listening Task (DLT) were administered under single-task and 
multitask conditions separately and in combination. Testing procedures were as 
follows: [Ref. 7] 

1) Psychomotor Test (PMT) 


¢ The PMT requires subjects to simultaneously center two visually 
presented cursors on fixed targets displayed on a CRT screen. The test 
performance measurements were the accumulated X, Y, and Z axis error 
distances between the target and the cursor point. A diagram of the 
experimental apparatus of Psychomotor Task (PMT) is depicted in Figure 
Bale 


2)  Dhichotic Listening Task (DLT) 


¢ The DLT first requires subjects to attend to auditory information 
presented to one ear, while ignoring similar information presented to the 
Opposite ear, and then after considering an auditory cue, to switch 
attention rapidly to the previously attended ear, or maintain attention to 
the previously attended channel. The modified Dichotic Listening task 
(DLT) [Ref. 9] consisted of letter-digit string sets of 24 listening trials for 


1 


2-POSITION 
SIMULATED KEYPAD 
OLT RESPONSE 
(Manual mode) 


TAPE RECOROER 
OLT RESPONSE 


(Voice mode) FOOT PEOAL CURSOR 


TARGET POSITION 
JOYSTICK 


FOOT PEDALS 


TAPE RECORDER 
(OLT Sttmui) 





Figure 3.1 Experimental Apparatus (adapted from Ref.7). 


each of 3 counter-balanced response modalities. Written, keypad and 
verbal response conditions were employed. The subjects were instructed 
to direct altention to one ear while ignoring the other, and correctly 
report the nine digits per trial. An example DLT trial is depicted in 
Figure 3.2 . The single-task performance measure for the DLT was 
simply the number of correct responses per 24 trials. 


3)  Multitask Performance 


e For the multitask condition, subjects performed two sessions of the DLT 
and PMT simultaneously. Performance measures for the PMT and DLT 
in the multitask conditions were identical to those of the single-task 
conditions except that multitask DLT performance was based on 12 
rather than 24 trials. 


The tests were performed in order such as: 
1) Psychomotor Test (PMT), 
2)  Dichotic Listening Task (DLT) and 
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PART I 


Pere Bar Res Neem GOB TFLERL 5 

“Right” (Vocal Channel “attend” Command) 

Right Ear Nese oe POF N 1 L 
PART II 

Betti ar BF4379 

“Left” (Vocal Channel “attend” Command) 

Right Ear Galler 6 2 


Figure 3.2. DLT Trial Examples (adapted from Ref.7). 


3) Multitask, that is, PMT and DLT simultaneously. 

To validate the test criteria, subjects’ flight performance were tracked until 
completing pilot training. Table 2 shows the mean and pass/fail correlation of each 
test measure. Correlations between each testing measure are presented in Table3. As 
you can see in Table 2 , testing measures 1, 5, 9, and 11 are significantly related to the 
flight training pass/fail criteria. Table 3 presents a correlation matrix of test measures. 

2. Psychomotor Test 

The USAF has utilized psychomotor testing in relation to pilot selection. The 
USAF primary objective has been an attempt to develop reliable and valid test devices 
and validation of test scores. In 1969, the Air Force Human Resource Laboratory 
(AFHRL) attempted to determine the reliability of two computer-based psychomotor 
tests (i.e., Two-Hand Coordination and Complex Coordination) [Ref. 8]. Both tests 
were employed to measure the subject’s ability to coordinate movements. Subjects 
were from Air Force Academy (AFA), Air Force Reserve Officer Training Corps 
(AFROTC), and Officer Training School (OTS). The largest group of the sample was 
AFROTC and additional subjects from Williams AFB were used for validation of 


testing results. 


TABLE 2 


DESCRIPTIVE STATISTICS FOR SINGLE AND MULTITASK 
MEASURES ANDISEEECTION TEsis 


(percent correct) 


Pass/Fail 
Test Measure Mean (r) 


1. PMT Session 1, X,Y,Z error combined 19116.36 -.283 
2. PMT Session 2, X,Y,Z error combined 11541.20 -.028 
3. PMT Session 3, X,Y,Z error combined 9896.90 -.24]1 
4. PMT Session 4, X,Y,Z error combined 9550.82 -.120 
5. DLT written response, No. correct 211.48 (97.9%) 360 
6. DLT keypad response, No. correct 210.60 (97.5%) 162 
7. DLT vocal response, No. correct 207.52 (96.1%) 244 
8. PMT X,Y,Z error (with keypad DLT) 21816.92 036 
9. DLT keypad response (with PMT) 102.34 (94.8%) 413 
10. PMT X,Y,Z error (with vocal DLT) 14038.10 -.002 
11. DLT vocal response (with PMT) 101.44 (93.9%) 95 
12. AQT standard score 5.66 see 
13. FAR standard score 6.80 361 


** Measure 1-7  : single-task 
** Measure 8-11 : multitask 
** Measure 12,13 : selection test 


(From Reference 7) 





Testing procedures were as follows: [Ref. 8] 
1) Two-Hand Coordination 


e The first of the tests presented a triangular-shaped target and a cross- 
shaped pipper on the CRT (Figure 3.3). The computer moved the target 
in an elliptical path and with varying speeds - faster near the 4 oclock 
and slower near the 11 o’clock position. The subjects were instructed to 
use both joysticks simultaneously (one is for up-down and the other is for 
left-right) in a coordinated manner to move the pipper, keeping it as close 
as possible to the target on the CRT. The computer measured absolute 
distance from the pipper to the target. The horizontal error (X,) and 
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TABLE 3 


CORRELATION MATRIX FOR SINGLE AND MULTITASK 
MEASURES AND SELECTION TESTS 


Measure 1 3 4 5 6 i 8 9 Oly 


401 

727 1544 

foe OG 7 sais 

-.626 -.131 -.694 -.443 

-.616 -.490 -.523 -.453  .540 

-.494 -.182 -.407 -.325 .708 .544 

Soe oO milly | 96) +503 «012 

-.708 -.244 -.782 -.508 .869 .588 .563 -.348 

526 .298 .532 .443 -.531 -.468 -.669 .873 -.445 

-.737 -.243 -.732 -.457 .913 540 .644 -.363 .887 -.529 
-.157 -.067 .060 -.021 .058 .104 .147 048 .116 .037 .058 
-.368 -.239 -.223 -.287 .050 .144 .223 .254 .144 .230 .099 .511 


** See Table 2 for description of numbered variables. 


(From Reference 7) 





vertical error (Y,) measurement were accumulated during 5-min test 
period. 


2) Complex Coordination 


e The second test presented a set of cross-hairs centered on the CRT, a dot- 
shaped pipper, and a thin vertical bar at the bottom of the CRT (Figure 
3.4). The subject controlled the pipper, both horizontally and vertically, 
using the floor-mounted joystick. The control responses were the reverse 
of what is traditionally required on aircraft to reduce the advantage of a 
subject with prior flying experience. The subjects were instructed to use 
the joystick to center the pipper horizontally and vertically on the 
intersection of the cross-hairs and simultaneously press the appropriate 
rudder-style pedal to center the “rudder bar” over the lower part of the 
vertical cross-hair. The computer measured in inches and accumulated 
during 5-min test period the the absolute distance from the pipper to the 
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PSYCHOMOTOR TEST I 


LEFT — RIGHT HAND COORDINATION 
(10 minutes) 


ae NN 
TARGET —F . 


PIPPER ; 





Figure 3.3 Two-Hand Coordination Test Depiction (adapted from Ref.7). 


intersection of the cross-hairs and from the rudder bar to the vertical 
cross-hair. The three scores obtained from this test were the cumulative 
horizontal error (X4), vertical error (Y>), and horizontal error for rudder 
bar (Z>). 


Table 4 presents five error scores (X), Yy) X47; Y> ZL) for each subject. 
These data show that the Two-Hand Coordination test was more difficult than the 
Complex Coordination. There might also be some common information about the 
subjects since all five test scores measured psychomotor ability. Table 5 shows 
intercorrelations across the tests. Two psychomotor tests were measuring different 
abilities with very little in common between the tests as indicated in Table 5. To 
validate the psychomotor testing scores, the mean scores of UPT graduates and 


eliminees were compared following completion of flight training. Table 6 presents the 
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PSYCHOMOTOR TEST I 


STICK & RUDDER SKILLS 


(10 minutes) 


3 PIPPER 


<< RUDDER BAR 





Figure 3.4 Complex Coordination Test Depiction (adapted from Ref.7). 


mean of the psychomotor scores among three categories: UPT graduates, all UPT 
eliminees, and UPT flving deficiency eliminees. All five scores had significant 
differences between graduates and either category of eliminees. Note that for all three 
of the Complex scores (X5, Y>, and Z5) were significantly worse than the graduates’ 
means. These results were used to identify subjects more likely to be fighter pulots. 
Figure 3.5 presents this result of all five means for both fighter-recommended (FR) 
and non-FR students. The FR students had significantly better scores than the non- 
FR students. The psychomotor screening equation was finally developed and the most 
useful screening score based on psychomotor ability can be computed from the 


following equation [Ref. 8]: 


ZS 


TABLE 4 
DESCRIPTIVE STATISTICS 


(inch) 


Test scone Range 


Two-Hand Coordination 
X1 (Horizon) 14,709 428 - 65,268 
Y 1 (Vertical) 16,748 5,687 - 57,292 
Complex Coordination 
X2 (Horizon) 4,514 326 - 71,040 
Y2 (Vertical) 3,860 486 - 76,320 
Z2 (Rudder) 5,580 167 - 71,040 
( From References) 


TAGE Es 
CORRELATION MATRIX 


X1 (Horizon) 
Y1 (Vertical) 
X2 (Horizon) 
Y2 (Vertical) 
Z2 (Rudder) 
(From Reference 8) 





Le =e (b, x X1;) ate (b, x i) 


Where Y; = psychomotor screening score of ith subject 
constant (0.95648) 


a 
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TABLE 6 
MEAN PSYCHOMOTOR SCORES BY UPT OUTCOME 
Xy “ X>5 Y> Z4 
Graduate Means (N = 1,348) 14,315 16,341 3,559 2,058 eS 
All Elim. Means (N = 377) — 15,829 I 7621 4,936 4,173 6,678 
Flying Deficiency (FD) 
Elim. Means (N = 199) 


(From Reference 8) 


16,302 18,007 DoS 4,702 7,580 


b, = weight of psychomotor X1 score (-.0000075748) 
b, = weight of psychomotor Y2 score (-.000020375) 
X1; = psychomotor X, score of ith subject 

Y2, = psychomotor Y> score of ith subject 


This equation provides a screening score based on psychomotor ability. These scores 
could be used in a Selection system by rank ordering; and the best candidates could be 
selected to fill the required training quotas. 
3. Integrated Selection System 

Although psychomotor testing scores are significantly related to pass/fail 
criteria in UPT, the ideal pilot selection method would consider all available 
informations on the subjects. 

Current operational USAF pilot selection criteria consider information from 
the five composite scores of the AFOQT, age at the start of UPT, possession of a 
college technical degree, and possession of a private pilot’s license or completion of a 
USAF light aircraft flying program [Ref. 8]. Table 7 lists the correlations of all 
available measures with the UPT pass/fail criterion for all reasons (FY79 to FY83). 

The integrated model was developed based on the USAF psychomotor testing. 
Because pilot candidates are chosen differently, depending on their source of 
commission, three different Integrated Pilot Candidate Selection Models (IPCSMs) 
were developed. 

The three [PCSMs equations were developed and will produce a score 
corresponding to the probability of UPT success using the following equations : 


E> 


MEAN PSYCHOMOTOR SCORE 


TEST Wi 


TESTA 


Figure 3.5 Mean Psychomotor Scores by FR/non-FR. 
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TABLE 
CORRELATIONS OF [PCSM PREDICTIONS WITH UPT OUTCOME 


Predictors Correlation with UPT Sample Sizes 
Pass/Fail 
Se Pass’ Fail alo 1,534 
FSP Final Grade 2A 1,534 
FSP Sum of Grades 303 poo 
AFOQT Pilot Score Sis 4,460 
AFOQT Navigator Tech Score 148 4,460 
AFOQT Academic Score O80 Ae 
AFOQT Verbal Score 007 4,576 
AFOQT Quantitative Score oll Bre 4,577 
Psychomotor XI Score -.118 1,918 
Psychomotor YI Score -.099 1,919 
Psychomotor X2 Score -.153 1,906 
Psychomotor Y2 Score -. 181 1,910 
Psvchomotor Z2 Score -.146 1,916 
Age -.120 8,438 
Gender 033 8.438 
Race 110 8,292 
Possession of Tech Degree Jul 8,183 


(From Reference 8) 


1) IPCSM I (OTS Model) 


l 


eed) ot (b, x X1;) Te (b, x i Se (b, x age) 


+ (b, x AFOQTP.) + (b; X AFOQTN,) 
+ (b, x FSP1.) + (b, x FSP2;) 


Where Y; IPCSM I score for ith subject 
a constant (0.10192) 
b, weight for psychomotor X1 score (-.000015211) 


oe 


= weight for psychomotor Y2 score (-.000010618) 

weight for age of subject (-.0012) 

weight for AFOQTP/AFOQT pilot score (0.00112) 

weight for AFOQTN/AFOQT navigator score (0.00029067) 
weight for FSP1/FSP final grade (0.08249) 

weight for FSP2/FSP sum of grades (0.01923) 


ale = psychomotor X1 score of ith subject 

be. = psychomotor Y2 score of ith subject 

age: = age at start of UPT of ith subject 

AFOQTP, = AFOQT pilot composite score of ith subject 
AFOQTN, = AFOQT navigator composite score of ith subject 
ese le = FSP final grade of ith subject 

Por? = FSP sum of grade of ith subject 


2) IPCSM If (AFROTC Model) 


where 


1 


Yj 


Tos Of 
vA k WN 


Y, =a +(b, x XI.) + (b, x Y2,) + (by x ages) 


+ (by X AFOQTP.) + (bs X AFOQTN;) 


= IPCSM II score for ith subject 


constant (1.72485) 

weight for psychomotor X1 score (-.0000030409) 
weight for psychomotor Y2 score (-.000022526) 
weight for age of subject (-.04517) 

weight for AFOQTP/AFOQT pilot score (0.00124) 


= weight for AFOQTN/AFOQT navigator score (0.00225) 


3) IPCSM III (AFA Model) 


where 


1 


Y¥; = a+ (b, X X13) © (Dy S12 ace (One ete crm 


IPCSM III score of ith subject 

constant (1.43056) 

weight for psychomotor X1 score (-.0000050818) 

weight for psychomotor Y2 score (-.0000072309) 

weight for age of subject (-.02111) 

weight for possession of technical degree (.08756) 
1 if ith subject has a technical degree, 0 otherwise 


28 


GRADUATES 









: ELIMINEES 

© 

° 
LJ 
© 
CO 
©) 
Dg 
pa 
YY) 
o 
a 
Ze 
eT 7. 

Hyer: 
; TYAN 
UAW 
| | 
IFCSM | IPCSM Il IPCSM Jl 
CORRELATION = 456 297 205 


Figure 3.6 Mean IPCSM Scores by UPT outcome. 
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The mean of the IPCSM scores are graphically presented in Figure 3.6. The 
differences between the means were significant and generated the better prediction than 
using the psychomotor test only. One way to select candidates is to accept only those 
candidates who obtain a screening score above a predetermined cutoff. The cutoff 
score would be determined by the quota required and attrition rate. The higher the 
cutoff score is set, the greater the impact on attrition in pilot training program. The 
FY83 UPT sample was used to cross-validate the three IPCSMs and resulted in the 
IPCSM scores were significantly correlated with UPT outcome. Since the “Whole 
Person Concept” is imbedded in this multi-dimensional IPCSM approach to pilot 
selection, the integrated selection method has better predictors than the psychomotor 
test only does. 

As we can see through the Chapter, all three pilot selection methods include a 
psychomotor testing which means psychomotor testing scores are significantly different 
between the successes and the failures. It should have the same result if psychomotor 
tests were applied to ROKAF. The only thing different between U.S. and ROKAF 
would be the cutoff score as a different required quota. The ROKAF decision maker, 
however, must know the advantages of using an experimental testing method in terms 
of their utilities or cost effectiveness. Chapter IV provides one way to make a decision 


and evaluate the outcomes. 
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IV. EVALUATION OF TESTING METHOD 


This chapter discusses how to make appropriate decisions which maximize an 
overall Measure of Effectiveness (MOE) or Uulity which might be used in a pilot 
selection program. The final goal of personnel selection programs has traditionally 
been to achieve maximum accuracy of measurement and efficiency of prediction. 
Decision theory recognizes that the outcomes of predictions are of primary importance 
to individuals and organizations. As discussed in Chapter III, for example, Korean 
AFA cadets are commissioned as officers after four years of attendance. Some fail 
either in undergraduate or operational flight training. This results in high attrition 
cost, dissatisfaction, lowered morale, and a number of officers totally unneeded in the 
service. Since the measurement and prediction are simply technical components of a 
system designed to make decisions, we have to define the MOE before making any 
decisions. There are, however, a number of possible MOEs in most decision problems. 
For example, there would be three possible MOEs in a pilot selection problem: 

1) Costs - selection, attrition and training, 
2) Pilot quality, and 
3) Probability that the quota will be satisfied. 

Thus decision outcomes should be evaluated in terms of their consequences for 
individuals and organizations. That is, in terms of their overall MOE or Utility. This 
provides us with a consistent way in which to combine different figures of merit or 
subjective and judgemental factors, and probabilistic aspects of outcomes into an 
overall MOE. 

To construct and evaluate a decision model, we have to find parameters which 
function in pilot selection outcomes. The major factors are [Ref. 10]: 


a) the base rate (the proportion of persons currently accepted who are 
satisfactory) 


b) the selection ratio (the proportion of applicants accepted) and 
c) the validity coefficient (the predictor criterion correlation). 
Taylor and Russell (Ref. 10] published a series of tables which illustrated that the 


value of a test was a function of the three considerations listed above. 


a 


A. BASE RATE (BR) 


UO , 
UNSATISFACTORY —>1<—— SATISFACTORY ——>] 
oy 


a 


TEST SCORE mMIGe 


LOW 
pe REJECTED ——-—- ee —— SELECTED) ———_»} 
Cc . 


Figure 4.1 Four possible outcomes of personnel selection (adapted from Ref.11). 


The possible outcomes of personnel selection programs are illustrated in Figure 
4.1. This figure depicts a bivariate plot for a selection test with a validity of .50. The 
horizontal axis, BB’, represents a particular base rate of satisfactory criterion 
performance, in this case, 50 %. The vertical axis, CC’, denotes the cutting score for a 
50 % selection ratio. Thus, Quadrant A contains a number of satisfactory subjects 
above the cutting score (valid positives, VP), Quadrant B contains unsatisfactory 
subjects above the cutting score (false positives, FP), Quadrant C contains 
unsatisfactory subjects below the cutting score (valid negatives, VN), and Quadrant D 
contains satisfactory subjects below the cutting score (false negatives, FN). [Ref. 11] 


Once the cutoff score is set, a definite number of predictions of success and failure are 


BZ 


made. As Figure 4.1 shows, the total number of predictive positives, Nop: may be 
calculated from : 


Nop = VP eee 


Similarly, the total number of predictive negatives, N__, is given by: 


pi’ 


Non 7. FN + VN 


The total number of actual positives, Nm , 1S given by: 


Pp 


ee ian 


And the total number of actual negatives, N._, is calculated by: 


an’ 


IN acer VIN oe all 
The probability of a valid positives, P(VP), 1s thus: 
P(VP) = VP/(Nap eee ee ETN 


Where TN is the total number of candidates. Figure 4.2 presents the probabilities and 
their relationships. 

When outcomes are expressed as probabilities they are referred to as “hit rates”. 
The probabilities or proportions of actual positives that exist in the total sample of 
candidates is called the “Base Rate” (BR).° The probability may be calculated directly 
BR = Nap/ TN or from the probabilities of two outcomes BR = P(FN) + P(VP) as in 
Figure 4.2 . The BR has reference to the base rate of the positive class and so the 
probability or base rate of negative in the total sample is given: | — BR. 

Selection measures are most useful when BRs are about .50 [Ref. 10]. As the 
BR departs radically in either direction from .50, the benefit of an additional predictor 


becomes questionable. 


*The proportion of candidates who would be successful in flight training without 
use of the test. 
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Test Diagnosis 
Actual Diagnosis Total 
Positive Negative 








































P(VP) = VP/TN P(FN)=FN/TN BR = Nap! TN 
valid positive rate false negative rate Base Rate 
Positive (Proportion of (Proportion of (Probability of 
positives called positives called actual positives) 
positive) negative) 
P(EP) [ene win P(VN) = VN/TN 
false positive rate valid negative rate 
Negative (Proportion of (Proportion of (Probability of 
negatives called negatives called actual negatives) 
positive) negative) 
SR= Nop (Npp t Non) 1-SR= 
Selection Ratio Non/Npp * Non) 
Total (Probability of (Probability of 


predicted positives) predicted negatives) 





Figure 4.2 Outcomes Expressed as Probabilities. 


B. SELECTION RATIO (SR) 

The probability or proportion of predicted positives among the total sample of 
candidates is called the “Selection Ratio” (SR).* This probability may be calculated 
directly as follows: 


SR=N 


pp <* 


PP i pn) 


*The proportion of candidates selected with the use of test. 
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or from the probabilities of two outcomes such that - 


SR = P(VP) + P(FP) 


Whenever a quota exists on the total number of candidates that mav be 
accepted, the selection rate becomes more important. As the SR approaches 1.0, for 
example, which means all candidates must be selected, it will be dominated by other 
alternatives and as the SR approaches Zero, it dominates all other alternatives which 
means a decision maker can afford to be selective. Cronbach and Glesser [Ref. 12] 
have shown that when the frequency distributions of the two groups (accept/reject) are 
plotted separately along the same base line, the optimum cutting score for 
distinguishing between the two groups will occur at that point where the two 
distributions intersect. However, in most practical selection situations, the decision 


maker is not free to vary selection ratios. 


C. VALIDITY COEFFICIENT 

Evaluating the benefit obtained from tests is of considerable practical interest and 
has been a problem in industrial psychology. The relation of benefit to validity has 
long been regarded as an important question. Most attempts to evaluate benefits have 
focused on the validity coefficient. As at least five formulas for interpreting the validity 
coefficient have been given, we face the problem of choosing between them or of 
modifying them. [Ref. 12] 

The validity coefficient is usually defined as the correlation of test score with 
outcome or criterion score. We shall use "yy in our discussion because the decision 
maker is interested in predicting the evaluated outcome, that 1s, the outcome expressed 
in utility units. The rule having the longest history translates the validity coefficient 


yy into the “index of forecasting efficiency” [Ref. 12], 


E a= (meme | el 


Although the validity coefficient r,, is a useful index, there are many ways in 


Xy 
which it falls short of accurately predicting success. From a practical point of view, the 


number of correct decisions made by a psychological test or assessment is more 


a 


important than the degree of association which exists between predicted and obtained 
scores [Ref. 12]. When two variables are expressed in dichotomous form (success- 
failure, select-reject) as in Figure 4.1 , the correlation between them is given by the 
point-biserial te coefficient”. A simplified computational formula for the r 5 coefficient 
is: | etal 


P(VP) - BR x SR 
r — rr ae nee a Say 
P [BR(1 — BR)SR(I1 — SR)} 1/2 


The value of to is computed knowing only three values: 
1) The probability of a valid positive, P(VP), 
2) The base rate, BR, and 
3) The selection ratio, SR. 
The computational formula for the r ; coefficient can be solved for the probability 


of valid positives: 


P(VP) = BRX SR + rv BR(1 — BR)SR(1 — SR) 


This expression is especially useful in evaluating the outcomes of test prediction. 
Given the validity of a test ce the base rate, BR, and the proportion of candidates to 
be selected, SR, it is possible to completely specify the outcomes of predictions. 

Tavlor and Russell [Ref. 14] denounced the practice of evaluating the 
effectiveness of predictor variables solely on the basis of their validity coefficients. 
They prepared tables for determining the effectiveness of a selection test by estimating 
the proportion of selectees that are considered satisfactory when the test validity, base 


rate, and selection ratio are known or assumed. 


D. DECISION STRATEGIES 

There are mainly two different decision strategies which are single-stage 
(nonsequential) and multi-stage (sequential) (Ref. 12]. The major difference between 
these two strategies is when the terminal decision is made. The general decision 


process is illustrated schematically in Figure 4.3 . 
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Figure 4.3 Schematic View of a Decision Process (adapted from Ref.12). 


1. Single-Stage Strategy 
A test or test battery is emploved to all candidates. Based on a predetermined 
cutting score, the terminal decision is made. Individuals are either selected or rejected 
and no further decisions are made. This strategy corresponds to selecting candidates 
randomly or by a medical examination only in pilot selection. 
2. Multi-Stage Strategy 
Terminal decisions are made onlv from the view point of a decision maker in 
multi-stage strategy. The cutting score on a test battery may be used to make 
investigatory decisions. This decision calls for additional information, dictating what 
test or procedure will be used to gather that information. This then leads to a further 
decision. The investigatory decisions may continue through a number of stages of 
subsequent testing until terminal decisions are made regarding all applicants. This type 
of decision process is used to select pilot candidates using one or more experimental 


testing method until a decision maker is satisfying. 


a 


E. THE EXPECTED UTILITIES (EU) 

As discussed at the beginning of this Chapter, utility provides a consistent way to 
compare each different selection method. The comparison would be accomplished by 
expected utility, 1.e., maximizes the expected utility across the possible outcomes. If 
the four possible outcomes such that valid positive, false positive, valid negative and 


false negative have utility U One 


Moe sree 
utility can be described generally: 


yp and Us, respectively, then the expected 


EU = Uv x P(VP) + Us, x P(FP) + Uy, x P(VN) + Us P(E Ne 
where Uy is the utility of testing method employed. Under different decision strategies, 
the expected utility should be considered separately so that we can figure out which 
decision strategy is better with which testing method such that expected utility for all 
possible outcomes can be maximized. 

A number of different decision strategies can result from combining different 
testing procedures. In the ROKAF selection problem, consideration in given to two 
simple decision strategies, 1.e., single-stage decision with random or medical 
examination, or double-stage decision with medical and one experimental test. This is 
based on the assumption that ROKAF has only two alternatives. 

1. Single-Stage Decision with Random Selection 

If the Korean Air Force uses a random selection process, providing that the 
SR, BR, number of applicants N, then the expected utility is going to be: 


EU = Uy, X P(VP) + Up, X P(FP) + Uy, * POVN) Soci 


P 


Where P(VP) BR X SR since Ca 0 


P(FP) = Skene) 

P(FN) = BR — P(VP) 

P(VN) ="(P = BR} = PEP) = (1 S9SR) = 3 GaN ean 
Uy = 0 since no testing employed. 


Whenever the ROKAF has data available this calculation gives the ROKAF a 
consistent way to compare alternatives with their utilities. For the present effort, it is 
possible to simulate ROKAF data and evaluate the results. Suppose ROKAF data is 


as follows: 
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e SR = .6, BR = .5, N = 500, 3 
e utility of valid positive, in dollar, Uyp = $80,000, 
e utility of false positive, Us, = - $80,000, 
e utility of valid negative, ee = $50,000, 
e utility of false negative, ie = 0, 
NOW We can compute the expected utility of a random selection process when 
the ROKAF uses random selection of KAFA cadets; 


P(VP) = BR X SR =.5 X 6 = 33 
Piebie — SRa> (VR) = .6— .3 = .3 
P(FN) = BR — P(VP) = .5 — .3 = .2 
Been) — (1 = SRS PORN) = (1 —..6) = .2 =.2 
then 
BU = 80,000 x .3 — 80,000 <x .3 + 50,000 X .2+0 X .2 


$10,000 per candidate 


2. Single-Stage Decision with Medical Examination 
When a medical examination is employed the probability of valid positive 


becomes: 
P(VP) = PS X SR 


Where PS is proportion of satisfactory candidates among those selected by a test. 
Taylor and Russell developed a table for this PS value and Appendix C provides these 
values as different BR and SR which is similar to Tavlor and Russell tables but more 
appropriate for point-biserial correlation, ro by Abrahams, Alf and Wolfe [Ref. 10]. If 
es is .25, then the PS = .582 with SR = .6, BR = .5 at Table given in Appendix C. 
Assuming the utility of medical examination is $50 then the expected utility 


becomes: 
P(VP) = PS X SR = .582 X .6 = .3492 
PGaeye — SRP — POP) = 3603492 = .2508 
P(FN) = BR — P(VP) = .5 = .3492 = .1746 


Bo 


P(VN) = (1 — SR) — P(FN) = (1 — .6) —..1746 = .2254 


thus, 


EU 80,000 X .3492 — 80,000 xX .2508 + 50,000 x .2254 — 50 


$19,092 per candidate 


which is more than the previous random selection with the amount of $9,092 utility 
using only a medical examination. This result indicates that selection via the medical 
examination is more effective than random selection. In other words, the medical 
examination increases a certain amount of utility of possible personnel selection 
outcomes. 
3. Double-Stage Decision with Medical and an Experimental Test 

Since the terminal decision is made via medical and experimental test 
decisions, it is reasonable to select enough candidates at the first decision so as to meet 
the required quota at the second decision. In the second decision, the terminal decision 
is made concerning the provisionally accepted applicants on the basis of the first test. 
If we assume that SR = .7 in this case and a .8 from the table with the same BR 
then the value of PS equals .709. With the utility of the experimental test equal to $80, 


the calculation of the expected utility is, therefore, as follows: 


P(VP) = PS X SR = .709 X .7 = .4963 
P(FP) = SR = P(VP) = .7 = 4ae8 — = 2037 
P(FN) = BR — P(VP) = .5 — .4963 = .0037 
P(VN) = (1 — SR) — PCFN) = (1 — .7) — .0037 = .2963 
then 
EU = 80,000 x .4963 — 80,000 X .2037 + 50,000 X .2963 — 50 — 80 


$38,093 per candidate 
which is more than three times the random selection utility and more than $19,000 


utility per candidate via medical test only. This result shows that if the ROKAF 
changes the present pilot selection method and uses an experimental test method with 
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medical examination, the increased expected utility of that change is about $19,000 per 
candidate even if using small scale simulated data. 
In other words, the potential advantages in changing the present ROKAF 


pilot selection system are obviously considerable. 


FL CAPER MODEL EVALUATION 

To this point decision strategies have been discussed in terms of their utilities and 
advantages. Managers of personnel systems, however, justifiablely demand an estimate 
of the payoff, in dollars, which can be expected to result from the implementation of a 
proposed selection program. It might appear that, for a given test validity and base 
rate, a personnel manager always should lower the selection ratio (i.e, become more 
selective) this strategy may or may not be cost effective since lowering selection ratio 
causes expanding recruit and selection efforts to meet the required quota. 

The purpose of this section is again to evaluate whether the changes are still 
feasible in terms of cost-effectiveness by using the Cost of Attaining Personnel 
Requirements (CAPER) model. Sands [Ref. 11] defined the CAPER model as follows: 


e “The CAPER model is designed to evaluate the cost consequences of alternative 
recruiting-selection strategies. Specifically, the CAPER model determines an 
optimal recruiting-selection strategy for minimizing the estimated total cost of 
recruiting, selecting, inducting, and training a sufficient number of persons to 
meet a specified quota of satisfactory personnel. In addition, the CAPER 
model considers the cost of an erroneous acceptance (selecting a person for a 
training program who subsequently fails to graduate) and the cost of erroneous 
rejection (rejecting a person who would have succeeded if given the 
opportunity).” 


Suppose that the ROKAF General Staff provides specific information for pilot 
selection policy such that the quota, the base rate and cost data for selection, 
recruiting, induction, training, erroneous acceptance and rejections, etc. Example of 
input data for the ROKAF is provided in Table 8 [Ref 13] and shows that 500 of 1,000 
persons graduated, indicating a .500 base rate. Suppose the data was given as follows: 

e the cost of recruiting an individual is $4,000, 


e the cost of the present selection procedure (medical examination) 1s $150 
including the physicians’ salaries and laboratory fees, 


e the cost of administering and scoring the experimental test is $100, 
e the induction cost per individual is estimated as $150, 

e the cost for training is $3,500, 

e the cost of erroneous acceptance is set at $1,000, and 


e the cost of erroneous rejection is estimated as $800. 


4) 


TABLES 
EXAMPLE CAPER MODEL INPUT DATA FOR ROKAF 







Graduates Failures 


Qualified for Acceptance 


Score Frequency| Number | Proportion | Frequency | Number 


Cuting Qualified for Acceptance 


Proportion 


0 1.000 
l 1.000 
Z .986 
3 .966 
4 Jaz 
5 .874 
6 .784 
7 .668 
8 ee 
9 388 
10 256 
[El LZ 
12 O80 
13 .038 
14 016 
iS 004 
16 .000 
i .000 
18 000 
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Given the simulated data, the ROKAF can plug into the CAPER model the 
equations and compute costs the present ROKAF selection process and the new 
selection process. The set of equations is used once for each possible cutting score on 
the new selection process. A cutting score of eight on the test will be used for 
illustration. 

Equation (1) gives the formula for estimating the number of applicants who must 


ibemecruited tm Ofder tO meet the quota : 
NR = Q/ [ (BRXPG,) | BRO, = 0 (1) 


mwinere NR is the number recruited, 
Q is the quota of satisfactory personnel, 
BR is the base rate, and 
PG, is the proportion of graduates who would qualify for acceptance at the ith 
cutting score on the test. 
Substituting the data of the present ROKAF selection procedure into Equation 
(1) gives: 


NR, = 50/ [ (0.50)(1.00) ] = 100 
Similarly, for the experimental selection procedure { i= 8): 


NR, = 50/ { (0.50)(0.92) ] = 109 


Equation (2) gives the formula for estimating the number of erroneous 


acceptances : 
NEA = NR(1 — BR)PF, (2) 


where NEA is the number of erroneous acceptances, 
PF. is the proportion of failures who would qualify for acceptance at the ith 
cutting score on the test, and the remaining symbols are defined above. 
Substituting the data of the present ROKAF selection procedure into Equation 
(2) gives: 
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NEA, = (100)(1 - 0.50)(1.00) = 50 
Similarly, for the experimental selection procedure ( ;= 8): 


NEA, = (109)(1 — 0.50)(0.532) = 29 


Equation (3) gives the formula for estimating the number of erroneous rejections: 
NER = (NR) BR) — PG, ) (3) 
Where NER is the number of erroneous rejections, and the remaining symbols are 
defined above. 
Substituting the data of the present ROKAF selection procedure into Equation 
(3) gives: 
NER, = (100)(0.50)(1 — 1.00) = 0 


Similarly, for the experimental selection procedure ( i= 8): 


NER, = (109)(0.50)(1 — 0.92) = 4 


Equation (4) gives the formula for estimating the number of persons who will 
accepted :; 


NA = Q + NEA (4) 
where NA is the number accepted and the others are the same as above. 
Substituting the data of the present ROKAF selection procedure into Equation 
(4) gives: 


NA, = 30 + 50 = 100 


Similarly, for the experimental selection procedure ( i= 8): 
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NA, = 50 + 29 = 79 


The total cost of the present selection procedure and the experimental selection 
procedure should be calculated by separate equation. 
Equation (Sa) gives the formula for estimating the total cost of employing the 


present selection procedure to meet the quota of satisfactory personnel : 


TC, = [(NRXCR)] + LCNRXCP) | +L CSAXCD) |] 
+ [| (NA\CT) ] + [ (NEA)(CEA) + (NER)\(CER) | (Sa) 


where TC, is the total cost of using the present selection procedure to meet the quota, 

CR is the cost of recruiting a person, 

CP is the cost of administering the present procedure, 

CI is the cost of inducting a person, 

CT is the cost of training a person, 

CEA is the cost of an erroneous acceptance, 

CER is the cost of an erroneous rejection, and the others are the same as above. 

Substituting the data of the present ROKAF selection procedure into Equation 

(Sa) gives: 


TC, = [ (100)($4000) } + [ (100)($1500) ] + [ (100)(S100) } 
+ { (100)($35000) ] + [ (50)(S8000) + (0)($1500) | 
= $3,619,120 


Equation (5b) gives the formula for estimating the total cost of employing the 


experimental selection procedure to meet the quota: 


ome Gh) tet ONRGGr) Je (NRX CE)] > { (NACI) | 
+ [ (NAY\(CT) ] + [ (NEAX(CEA) + (NER\CER) ] (Sb) 


where TC, is the total cost of using the experimental selection procedure to meet the 
quota, CE is the cost of administering the experimental procedure, and the others are 
the same as above. 


Using Equation (5b) for the experimental selection procedure ( i= 8) gives: 
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TC, = [ (109)($4000) | + [ (109)($1500) } + [ (109)(S80) ] + [ (79)($100) ] 
+ [ (79)($35000) | + [ (29)($8000) + (4)$1500) | 
= $751,920 


A useful program for various data values is provided in Appendix B and creates a 
sample CAPER model output data for ROKAF in Table 9 for the different values. 
This output contains five types of information : 

1) number recruited, 
2) number of erroneous acceptance, 
3) | number of erroneous rejections, 
4) number accepted, 
3S) Wotalcost: 
These five estimates may suffice for many personnel program managers. And also 
contains cost for each of the following: 
1) recruiting 
2) selection 
3) induction 
4) training, and 
5) erroneous decisions. 

Table 9 shows that as the cutting score increases the subsequent results are as 

follows : 

1) a greater number of persons must be recruited, 

oy) a smaller number of persons is accepted, 

3) the number of erroneous acceptances decreases, and 

4) there 1s an increase in erroneous rejections. 
These four consequences have cost complications. As recruiting and selection costs 
increase, induction and training costs are decreased. The cost of erroneous acceptances 
and erroneous rejections, decreases at first, hits the cutting score that minimizes the 
sum of both costs ( i= I!) and then increases as the cutting score is raised farther and 
the total cost changes similarly. Figure 4.4 graphically shows this changes of total cost 
as increasing the cutoff scores. 

If the ROKAF wants to achieve 50 graduates at a minimum total cost, 


comparing the estimated total cost of the present selection procedure ($4,460,000) in 
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TABLE 9 
EXAMPLE OF CAPER MODEL OUTPUT DATA OF ROKAF 


+ + 
PSeurT NO NO | NO NO | 
eer | REC. , ACC. | ERR. | ERR. | COST , 
i ase ee Owen eee een we we OOO ee ee ee we Oe ee we ewe ee ee www www ee meme mem ee ew ew ew eee ee ew wwe 

| | | Pence. | REI .~{ RECRUIT || SELECT eal INDUCT {| TRAINING | ERR.DEC. | TOTAL | 
> 7 
nee 100 | +4100 | S10) || 0 | 400000. | 150000. | 10000. | 3500000. 7 400000. | 4460000. | 
| | | | | 

meee 100 6 |)~6100 (| 50 | 0 | 400000. | 158000. | 10000. | 3500000. | 400000. | 4468000. | 
meee Lod 6 |l)6hC6100 I Ei) || 0 |} 400000. | 158000. | 10000. | 3500000. | 400000. | 4468000. | 
meee ))~6=6C1 00 | 99 | 49 | 0 | G00000. | 158000. | 9900. | 3465000. | 392000. | 4429900. | 
aes 6=6CL OC 93 | 438 | 0 | 400000. | LPSse0005 | 9800. | 3430000. | 384000. | 4381800. | 
1 G@ { 100 | 97 | 47 | 0 | 400000. | 1523000. | S700 | 5595000). 4 576000, 1) 4556700. | 
er a 94 | 4G | 0 | 400000. | 156000. 4 9400. | 3290000. | 352000. | 4209400. | 
eos 102 | 90 | 40 | onl 468000. | Pes On| S000 meee sl SOCO0.) | 321500. | 4049660. | 
eee | = =—(104 =| 35° | 35 | 2 | 4160C0. | 169320. | 8500. | 2975900. | 283000. | 3844820. | 
mesee)6LO9 Toe 29 | 4 | 436000. | i 2220 7900. | 2765000. | 238000. | 3619120. | 
ieee |) | (CL1S i o5t|| 25 || 9 | 472000. | ' 186440. | 7300. | 2555000. | 197500. | 34138240. | 
ioe | «134 (| o7 | 17 | ie || §36000. | ell7Zz0 | 6700. | 2345000. | Lelsods | 3260920... | 
meee | «(16S | 62 | eat | S204 652000. | 257540... | 6200 -leeel 0000. | 144000. | 3229749. | 
meee | | 214 | 59 | 9 | Sil 856000. | S5e 120.7 S900. \ 206s000. | 1S, 500. | S422526- | 
enor | «(S01 | 56 | Sl) 01 | 1204090. | 475580. | 5600. | 1960000. | 199500. | 3844680. | 
1246 | 463 | 54 | G4 | 181 | 1652000. | 731540. | 5400. | 1890000. | 303500. | 4782440. | 
eo d|lCU7984 «Ci 525) 2 les areal 5176000. 3\) 1254520. || S200 lee cooo00.) | sooo 00.. || ~67 92220. | 
ime. | 1471 | 50 | 0 63855) 5884000. | 2324180. | S000 Wy50000- )] Voaz7500. | 1log99a6so. | 
1 17 | 2941 | 50 | 0 { 1421 | 11764000. | 4646780. | S000, Im 50000. | 2151500. | 202972380. | 
fee | 7143 | 50) | O |) 352) | 2385/2000. | 11285940. | 5000. | 1750000. | §281500. | 46894416. | 
+ + 


Table 9 suggest that using the experimental testing procedure with a cutting score of 8 
($3,619,120) would be cost effective which is the same result as the utility evaluation. 

Table 9 shows that the minimum total cost of attaining the quota, 50 graduates, 
is $3,229,740 using a cutting score 11 on the test and the optimal recruiting-selection 
Strategy is to recruit 163 persons. The best estimate is that 62 of these persons will 
qualify for acceptance and 12 of the selectees can be expected to fail the training 
course, leaving the 50 graduates required to meet the required quota. In comparison 
with the present selection procedure, the use of the experimental selection procedure 
will save an estimated $1,230,260 or $24,605.20 per graduate. 
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Figure 4.4 Graphical view of CAPER model output. 
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V. DISCUSSION 


The increasing number of pilot candidates entering the ROKAF has resulted in 
increasing costs due to high attrition rates and decreasing qualitv of pilots who do 
complete the program. Further, training funds and personnel resources remain limited. 
Solving these problems will not be an easy task. It appears that there is not a single 
simple solution. However, it is imperative that some restructuring of the selection 
process be initiated as soon as possible as the potential negative impact these problems 
will have continues to increase. [It is hoped that the current “Operations with 
Economic Efficiency (OEE) Program”, which is attempting to maximize the cost 
effectiveness of all ROKAF operations will be able to adequately address the pilot 
selection problems. This thesis represents an attempt to suggest an effective method 
for improving the selection process. 

As was discussed in Chapter I, other nations - France, Italy and England - 
initiated the selection testing efforts as early as World War I and the USN (USAF) 
reduced the attrition rate from 60% to 30% (25%) using psychological testing in the 
early 1970s. The Pensacola 1000 aviator study found that psychological and 
psychomotor measures had more validity for the prediction of success in flight training 
than did physiological measures. The psychomotor testing scores are significantly 
related to flight training pass/fail criteria. The ROKAF pilot selection system, 
however, only depends on actual flight performance. 

As we evaluated its potential benefits to the ROKAF in Chapter IV, the expected 
utilities and CAPER model, could have a significant impact on pilot selection, pilot 
training and aircraft accidents. In terms of the analytic CAPER model, where output 
data are fixed by the values of the input data, the utility of the CAPER output can be 
no better than the accuracy of the input data. If the cost estimates are inaccurate, cost 
forecasts and the optimal recruiting-selection strategy will be distorted. This may not 
be a serious problem since the approximations are usually adequate for the kinds of 
decisions that estimates are used to make, and there is no need for utility estimate 
down to the last dollar. 

It has been shown that a greater number of pilot candidates attrite during actual 


flight training than during the pre-training medical screening process. These 
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candidates, though often having the option to continue their military careers in other 
aviation related fields, frequently feel that they are inferior officers because of the 
failure during flight training. These feelings of low self esteem, morale and motivation 
often result in poor performance and the involuntary termination of the officer’s career. 
It is suggested that if a viable psychological test battery was administered in addition 
to medical screening and a candidate failed that test then (1) there would be a 
significant savings in actual flight training time for that candidate and (2) his being 
identified as “not aeronautically adaptable” might not have the psychological impact as 


having failed because of poor performance during actual training flights. 


A. RECOMMENDATIONS 

This study which is one of the OEE programs planned to reduce the cost and 
improve the efficiency of all ROKAF operations was essentially designed to emphasize 
the importance and necessity of mental testing methods on pilot selection for 
successfully screening out potential failures and suggests useful pilot screening methods 
to ROKAF. Also provided are evaluation techniques for decision makers which are 
the decision strategies and CAPER model. 

It is clear from the overall study results that the use of psychological tests or test 
batteries of other nations can improve the prediction outcomes of the ROKAF pilot 
candidate selection method. Many other national Air Forces have used psychomotor 
screening to select their pilot candidates and continue efforts to develop this 
methodology. Some of the psychomotor scores are significantly different between 
candidates who graduate from UPT and those who are eliminated. The ideal selection 
system, however, would measure and consider all relevant characteristics of a 
candidate. The UPT selection system currently used by the ROKAF does not consider 
all relevant information about candidates and many candidates fail training because of 
deficiencies in flying aptitude. The intense situation of ROKAF, moreover, requires 
selecting high quality student pilots and the candidates with the lowest probability of 
success would be screened out before training. The higher the cutoff point is set, the 
better the pilot quality, cost effectiveness not with standing. The “false rejection’, 
however, is a problem with all screening systems, including flight screening, and must 
be evaluated in terms of the percentage screened and the quality of the graduating 
students. 

As suggested in Chapter IV, a change using experimental testing on ROKAF 
pilot selection method is feasible. It suggests a savings of $24,605.20 per graduate in 
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our example. Multitask performance is useful for the primary flight training 
performance prediction. The experiment takes only 20 minutes and is easy to 
administer. The psychomotor screening system used by the USAF, which includes 
Two-Hand Coordination and Complex Coordination, has been shown to be useful in 
screening not only candidates for UPT but also for identifying those who demonstrate 
a potential for successfully completing advanced fighter training. Any of three 
Integrated Pilot Candidate Selection Models could be used to select UPT candidates in 
the same manner as the psychomotor screening system. The whole person concept is 
imbedded in the multidimensional IPCSM approach to pilot selection. 

The implementation of psychomotor screening and multitask performance will 
improve the quality of student candidates and reduce attritions in ROKAF UPT. 
Based on overall considerations, however, an integrated selection system including 
psychomotor screening is recommended for ROKAF. 

One of the preferable pre-selection systems that suggests for ROKAF pilot 
selection system of AFA cadets in this study would be: 


1) |The General Staff defines the number of pilots needed, and a circular is issued 
identifving the number of candidates required. 


2) The selection process includes: 
a) Preliminary Examination - Medical 
b) EEG recordings (Electroencephalogram) 
c) Integrated Psychological Test 
d) Individual interview. 


3) Cadets must wait until after their graduation from the KAFA and commission 
as officers before starting flight training. 


Other useful methods which might also be incorporated in a pre-selection screening 
process would be: 

e Perceptual/Cognitive Paper-and-Pencil Tests, 

e Neurological Test, 

e §6©Residual Attention Test. 
The Residual Attention Test could be especially valuable in predicting how well an 
individual will handle emergency situations. This could potentially have an impact on 
the aircraft accident rate in accidents due to human error, incorrect procedures or Just 
poor handling of the aircraft during emergencies. It is obvious that with the many 
types of individual psychological and psychomotor tests available that further research 


must now address which individual tests and combinations of tests and batteries of 
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must now address which individual tests and combinations of tests and batteries of 
tests should be incorporated into the ROKAF selection program. It is strongly 
recommended that a ROKAF pilot selection program research and development (R & 
[D) team be organized to address this and other selection issues. The team could also 
benefit greatly from a close liaison with U.S. Navy and Air Force R & D centers 
currently engaged in aircrew selection methodology. 

It is hoped that this study will be useful in stimulating the ROKAF in developing 
improved selection methods and in predicting success in training as a means of 


increasing aircrew training program cost effectiveness. 


a2 


ACTB 
AFA 
AFB 
AFHRL 
AFOQT 
AFROTC 
AOC 
AQT 
ASTB 
FAR 
FR 
FSP 
IPCSM 
KAFA 
OEE 
OTS 
ROKAF 
R&D 
UPT 
USAF 
USN 


List of Abbreviation 


Aircrew Classification Test Battery 

Air Force Academy 

Air Force Base 

Air Force Human Resources Laboratory (AFSC) 
Air Force Officer Qualifying Test 

Air Force Reserve Officer Training Corps 
Aviator Officer Candidate 

Aviation Qualification Test 

Aviator Selection Test Battery 

Flight Aptitude Rating 

Fighter Recommended 

Flight Screening Program (OTS) 
Integrated Pilot Candidate Selection Model 
Korea Air Force Academy 

Operations with Economic Efficiency 
Officer Training School 

Republic of Korea Air Force 

Research and Development 

Undergraduate Pilot Training 

United States Air Force 


United States Navy 
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APPENDIX A 
ATTRITION CATEGORIES 


A student pilot can be dropped from a training program for several reasons. It is 
agreed that there is no one simple causative factor nor a quick and easy solution. 
Some of the primary reasons for student attrition are : 

I. Voluntary Withdrawal or Drop On Request (DOR) 

Over 50 research reports and findings associate the main reason of DOR 
with anxiety, tension or fear. Most of the specific reasons for the DOR are expressed 
an anxiety or tension with regard to flying, fear of flying, or lack of confidence in 
handling aircraft. Also mentioned are a dislike or distaste of flying and a shift to a 
civilian career interest. Some of the researches indicate a connection of fear of aircraft 
accidents or crashing with failure in training. 

2. Not Phisically Qualified 

Most research suggests that “a person with anxiety may see the flight 
surgeon when he has a symptom which he knows might result in his being dropped 
from the program; whereas, a nonanxious, highly motivated person might never go to 
the flight surgeon.” 

3. Flight Failure (FF) 

Figure 1.1 mentions the types of attrition by stage of training. The data 
provide dramatic evidence of the relationship of DOR and FF in primary training and 
the transition phase. 

The research literature generally supports the view that anxiety 1s both a 
cause and a symptom of inadequate or inferior performance in training. It can be 
broken down into two subcategories: 

1) A student group prone to anxiety or with anxiety predisposition. 
2) A student group going through a transient anxiety producing situation. 
4. Not Officer Material 

Is a statistic more oriented to the time during which attrition occurs rather 

than a specific type of attrition. 
5. Academic Failure 


Unsatisfactory performance in academic work. 
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6. Transfer to Another Training Program 
This is a small percentage of the total attrition and means a request to 
change one’s specialty. 
7. Practical Work Failure 


Unsatisfactory performance in a ground simulator or training device. 
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APPENDIX B 
CAPER MODEL PROGRAM 


KKAKKKKKKKKKKKKKKKKKKKKKRKKKKKRKKKKKKKKKKKEKKRKKKKKKKKKKKKEKEKKKKKKKKKKKK 


The sroar 
fone 
exec file. 


The key variables used are 


This program could be used 


k 
x 
k 
k 
x 
x 
x 
x 
x 
* 
x 
* 
x 
* 
x 
x 
x 
x 
x 
x 
x 
x 
* 
k 
x 
x 
ke 
* 
x 
x 
x 
x 
x 
x 
; some developments. 
PARAMETER (N=20 
REAL Ee ae 


DATA NOQ/50/,BR/.5/,C 
x CEA/8000/ ,CER/1 


*x** READ DATA FROM DATA FILE *** 


DO 10 I=1 
READ 
10 CONTINUE 


xkX QUTPUT FORMAT *** 


WRITE(3,15 
WRITE(3,65 
FORMAT (1X, | 
WRITE(3,75 


(1,5) PG(I),PF(I) 


65 
io 


CULMiae NO | 


WRITE Si) 5 
WRITE(3,25) 
WRITE (3 

<r Clee INDUCT 


CAPER MODEL 


This program provides a CAPER MODEL OUTPUT as various input data. 
am reads data from data file, CAPER DATA, and creates a 

output, CAPEROU LISTING on 
he data used aneminciums 
no meaning of ROKAF data but only for showing how the program works. 


NR : number recruited 

NA : number accepted 

NEA : number of erroneous acceptances 

NER : number of erroneous rejections 

NQ : the quota of satisfactory personnel 

BR) Sas cmucaue 

PG ¥: prepotclodsor peace who would qualify for 
acceptance sata ches tth CUCL Inga sconemolmmuecr 

PF : proportion of failures who would qualify for 
acceptance at the ith) cuteing scorencnmeccr 

CR : cost of recruiting a person 

CP : cost of administering the yome=entepiocedine 

CE : cost of administering the experimental procedure 

CI : e€ost of Imagvetind vceceuscn 

CT : cost of training a person 

CEA : cost of an erroneous acceptance 

CER: COSt of an €hroneous Tegeccron 

TCP <:etotal cost of uste the present selection 
rocedure to meet the quota 

ee otal cost of using the experimental selection 


procedure to meet 


changing only the data and also useful for interactive program with 


KRKEKEKKKAKKKKKAKKKKKKKEKEKEKKKKEKEKKKKKKKKRKKKKKKKRKKKKEKRKKKERKKRKKKKKKKKKKK 


| 
TRAINING | 


our disk automatically using 
e Table 8 and there is no 


h 


che quota 


* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
* 
7 
* 
* 
effectively for other CAPER MODEL js 
* 
* 
* 


Bo Ol ECO OO ae 


NO | NO NO 


| [',72X,'|') 


FORMAT 1X,! SCORE| REC. | ACC. | ERR. | ERR. |',34X,'COST',34X,'|' 


ACG. I 
ERR. DEC. 


| 
OTAL 


REJ SEL 


| RECRUIT | 


- ri 
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ei) 
WRITE(3,*) 


MAA eOAWCULATE TOTAL COST *** 
BOZO J—ien 
NR = NINT(NO/ (BR x Behe 
R * (1 - BR PF 


NEA a $43 
NER NINT(NR * BR * (1 = PG(J) 
NEA 


Ck 

GP 

CP + NR * CE 

or 

CT 

NEA * CEA + NER * CER 
St Le TE 

TCR S=—s1CS +°SF 
= TCS + SE 


: ae 1) THEN 
WRIT eae hie, NAWNEA, NEK,TReor,1i,1l,TE,1TCP 
WEE TE Gao 


Ww) 
try 
ee | | 
a a 
A 
+A A Ht + 


“-_— 
Cy 


E 
WRITE(3,45) (J-2),NR,NA,NEA,NER,TR,SE,TI,TT,TE,TCE 


END@is 
20 CONTINUE 


WRITE (3,15) 

FORMAT (F5.3,5X,F5.3) 

FORMAT 1K, Fs tOBt 

FORMAT ('+!' 737,72('= 

ae ! 
pee. g 


OP WNrR UO 
Mma 


FORMAT (1X, ' 
PORMAT( 1X," 
SLOP 
END 


; 
NA | a 
4 





x*k*X DATA FILE *** 
PG PF 


1,000 oo 
e000 1.000 
i000 1.000 
POUr 986 
evo “206 
1.000 Ja2 
2296 874 
984 . 184 
202 -668 
920 OZ 
848 388 
144 200 
-olZ aoe 
468 080 
oS 5 eRe: 
+216 -ULS 
126 004 
.068 000 
.034 000 
014 .000 
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APPENDIX C 
TAYLOR AND RUSSELL TABLES 


N. M. AsraHams, E. F. ALF, and J. H. Wotrs 


TABLE 1 


Prorogrion Wao Wrz Bz Sarrsracrory Amoneo THosz SELecrep, POR GIVEN VALUES OF 
tax Proportion or Present Exptoregs Consrpered SaTts¥ACToRY, TEE 
Seriecriom Rarro, anD Ponvr-Brszrraz r 





.00 050 .0S0 .050 050 0s0 .0S0 .0S0 0S0 .0S0 .050 .050 
.0S 076 .072 .066 063 061 059 .0S7 053 054 052 051 
.10 siz .099 .086 078 072 067 .063 060 .0S6 083 052 
1S 187 131 106 092 082 075 .068 063 058 084 052 
.20 216 169 128 107 092 082 073 066 .000 055 .052 
25 278 .210 151 Zt 101 087 076 068 .061 085 052 
30 352 .255 173 133 109 092 079 070 .062 055 .053 
35 £433 .302 194 144 115 .095 .081 .070 .062 036 053 
.40 519 348 | zie 183 119 097 .082 071 .062 .056 033 
45 607 393 ; 189 122 .099 083 07! .062 .0S6 053 
.50 694 431 | 238 163 124 .100 083 071 .062 .0S6 053 
$5 aie 462 .245 165 124 .100 083 071 062 | .056 053 
60 849 484 .248 166 125 .100 083 | .071 .062 056 053 
65 909 495 250 167 125 .100 .083 .071 .062 | 0356 083 
.70 98+ 499 250 167 125 .100 .083 .071 062 | .056 053 
73 982 500 250 167 125 .100 083 .071 .062 ! 056 053 
80 | .996 .500 .250 167 125 | .100 .083 O71 | .062 | .086 | 083 
‘as | 996 | .500 .250 167 125 | .100 .083 .071 | .062 036 | .053 
90 | 996 | 500 | 290 | .167 | 125 | 100 | .083 | .o71 | .062 | 086 | .083 
95 | 1.000 | .500 280 | .167 125 .100 083 071 | .062 056 055 
1.00 | 1.000 500 | .250 167 125 .100 | 083 071 .062 0356 05: 





Proportion of emplovees considered satisfactory = .10 





100 | .100 | .100 | .100 |} .100 | .100 | .100 | .100 | .100 





00-17 100) 2) setOO nie 
oS | 134 | 129 | 122 | 118] 215 | 012 | tO) 107 aioe tas toe 
AO} 176 | .162 | 147 | 137 | 130 | 124 | 118 | 11a | 109 | 105 | .103 
AS | 226 | 200 | .173 | 187 | 045 | 13S | 0127 | 120 | 113 | 107 | 104 
20] 284 | .243 | 202 | 177 | 160 | 1146 | 138 | 125 | 116 | 108 | .104 
25] 349 | 290 | 232 | 198 | 175 | 187 | 142 | 130 | 1119 | 109 | .104 
30] 422 | 343 | .266 | 219 | 189 | 166 | 148 | 133 | 121 | 110 | .105 
35 | 502 | 399 | 297 | 240 | 202 | 175 | .183 | 136 | .122 | .110 | 108 
40} 588 | 460 | 330 | 260 | 214 | 182 | 158 | 139 | 123 | ttn | .105 
45 {| 676 | $28 | 366 | .278 | 225 | 188 | 161 | 140 | 124 | 411 | .10S 
5O| .765 | $92 | 397 | 295 | 234 | 193 | 166 | 142 | 125 | 111 | .108 
SS} 847 | 661 | 427 | 309 | 240 | 196 | 165 | 142 | 128 | .1t1 | .108 
Oo} 917 | 731 | 454 | 320 | 2465 | 198 | 166 | 143 | 128 | ttl | 105 
65 966 800 476 327 248 199 166 143 125 111 105 
70} 991 | 864 | 490 | 331 | 249 | 200 | 167 | 143 | 128 | 111 | 108 
75] 999 | 920 | 497 | 333 | 250 | 200 | 167 | .143 | 125 | .t11 | 108 
80 | 1.000 | .964 | SOO | 333 / 280 | 200 | 167 | 143 | 125 | 111 | 105 
85 1.000 | .990 | 500 | 333 | 280 | 200 | 167 | .143 | 125 | «111 | .105 
50 | 1.000 | .999 | $00 | 333 | 250 | 200 | 167 | .143 | 125 | 111 | .108 
95 | 1.000 | .999 | $00 | 333 | 280 | 200 | .167 | .143 | 125 | 111 | .105 
1.00 | 000 S00 | 333 | 280 200 | 167 | 143 | 125 | 111 | 108 
ce peel recente nee ip 


1.000 | EE 
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TABLE 1—(Continued) 





Selection ratio 














Fp 
050 | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | 950 
Proportion of employees considered satisfactory = .20 
00 | .200 | .200 .200 .200 | .200 .200 .200 200 | 200 | .200 | .200 
05 | .244 | 237 | 229) 224 | 220 | .216 | 213 | .210 | 207 | 204 | 202 
My 2030) 2m | 260) 288) 230 | 232} 225 | 219 | .213 | .207 | .204 
15 | 347 | .320 | .292 | .274 | 260 | .248 | .238 | .228 | .219 | :210 | 205 
20| 405 | 368 | .326 | .300 | .280 | .264 | .249 | .236 | .224 | .213 | .207 
25 | .468 | 418 | 363 | .328 | 301 | .280 | .261 | .244 | .229 | .215 | .208 
Bonescsm) 47m deme, 356 | 322 | 295°| 2728) 253 | .234 | 217 | .208 
35 | .605 529) 441 | 385 | 4 | SUL | 283 | 259 | 238 | 218 | 209 
40 | .676 | 589 | 484 416 | 366 | .326 | .293 | .265 | .241 220, | 210 
AS | 747 | 652 | .529 | 448 | 387 | 340 | 302 | .270 | .244 | .220 | 210 
50| .8i5 | .717 | .577 | .480 | ‘409 | [354 | 310 | (275 | (246 | a1 |. -210 
55} 876 | 782; 627 | S14 430 | 367 | 318 | 279 | 248 | 222 | 210 
| 927 | .846 | 680 | 547 | 450 | 3738 | 324 | 282 | 249 | .222 | .210 
65 | .964 : 904 | .736 | 581 | 468 | 387 | 328 | 284 | 249 | 222 | 210 
70 | .986 | .952 | .794 } 612 | 482 | 304 | 331 | 285 | .250 | 222 | .210 
75) 996 | .983 | 852 639 | 492 | 398 333) 286 | 250 | 222 | 210 
80 | 1.000 997 | 909 | .657 | 498 | 400 ly 33a) | 286 ff 250 Wo .222)i|) .210 
85 | 1.000 | 1.000 | 958 | 665 | S00 | 400 | 333 | 286 | 250 | .222 210 
30 | 1.000 ; 1.000 | 990 ' 667 | .500 400 | 333 | .286 | .250 | .222 | .210 
95 1.000 ° 1.000.990 667 | 300 | 400 | 333 | 286; 250 | 222, 210 
1.00 1.000 | 1.000 | 1.000 | 667 | 500 400 | 333,286 | 250 ) 222 | 211 
Proportion of employees considered satisfactory = .30 
oo! 300 | 300! .300 | 300 | 300 | 300 | .300 | 300 | 300 | 300 | 300 
Seetsome Ge Stel aime) G22) «Sige || asi | «308 | 304 | 302 
to} 401 385 | 367,’ .354 | 345 | 337 | 329 322 | 316 | 308 . .305 
15] 486° 430 | 402 | 383 | 368 | 355 | 344° 333 | 323 | 312 | 307 
20) 513478 | 438) 412 | 391 | B74 | 358 HE 330) 316 | 308 
oi .5i2 | 528 | 476 442 415 393 372 | 354 | 336 319) .310 
30! 032 | 379; 516 | 473 | 440 | 412 | 387 | 364 | 343 | 322) 311 
35 692 ' 632 | 1557 | 1506 | 465 | .431 | 401 | 1374} 349 | 324. 312 
4 751 | 68 | .600 | 540 |} .492 | .451 | .415 | 383 | 354m 327) sid 
45 | .808 .740 646 .576 519 471 429 392 .359 329 314 
50 | 860 | .794 | .693 | 614 | 547 | 491 443 | 401 | 363 | 330 | 315 
SS} 905 | 846 ) .742 | 653 | 577 | 512 | 456 | 408 | 367 | .331 | 315 
60 | 942 | .894 | .794 | 696 | 608 | .532 ) 468 | 415 | .370 | .332 : 316 
65 | .970 | 936 | 846 | .741 | 639 | 1552 | 479 | 420 | 372 | .333 316 
. 987 | 967 .897 .789 .671 .570 | 488 424 374 | 333, 316 
75 | .996 ; .988 944 .839 : .702 584 .495 427 | 375 333.316 
80 | 999 | .997 | .979 891 | .728 | 594 | 498 | 428 | 375 | 333.316 
85 | 1.000 | 1.000 | oor | 941 | 745 | 599 | S00 | 428° 375 | 333, 316 
90, 1.000 | 1.000 ; 1.000 ; .982 | .750 600 | .500 | .429 375 | 333 | 316 
9s | 1.000 | 1.000 | 1.000 | .982 | .780 | .600 | soo | .429 | .375 | .333 | 316 
1.00 | 1.000 | 1.000 | 1.000 | 1.000 | 750 | .600 | 500 | 429 | 375 | 333 | 316 
Proportion of employees considered satisfactory = .40 
00 | .400 | .400 | .400 |; .400 | .400 | .400 | .400 | .400 | .400 | .400 | .400 
os| 451 | .444 | .435 | .429 | 1424 | .420 | .416 | .412 | .408 | 405 | .403 
10] .504 | .488 | .470 | 458 | .448 | 439 | 432 | 424 | 417 | .409 '  .405 
1s | .ss7 | .534 | .506 | .487 | .472 | .459 | 447 | 436 | .425 | 414 .408 
20 | ou | 580 {| 542 | 517 | 497 | 479 | 463 | 448 | 433 | 418 410 





N. M. ABRAHAMS, E. F. AF, anp J. H. WoLre 


TABLE 1—(Continued) 


Selection ratio 


























Tob 
050 | 100 | 200 | 300 | 400 | 500 | 700 | 800 | 900 | 
Proportion of emplovees considered satisfactory = .40 
25 | .665 626 580 548 522 500 479 460 441 422 412 
300| 71a 673 618 580 548 521 496 472 449 426 414 
35 | .768 720 658 613 375 542 512 484 456 429 415 
40 | .816 766 698 647 603 564 529 496 464 433 417 
45 | .860 811 740 682 632 588 546 508 AT1 436 A418 
50 | .900 854 782 .720 663 612 564 519 477 438 419 
53 | .933 894 824 758 696 637 582 530 483 440 .420 
60} .959 929 866 799 .730 663 .600 S541 489 442 420 
65 | .978 958 906 842 768 691 617 551 493 443 ao 
70 | .990 979 943 885 807 719 634 560 496 444 421 
758 | 997 992 972 .928 850 748 649 566 498 444 AG 
80 | .999 .998 991 966 894 aga 660 570 500 444 oe 
85 | 1.000 | 1.000 .998 992 939 792 665 571 500 Att 421 
90 | 1.000 | 1.000 | 1.000 | 1.000 979 .800 667 571 500 44 421 
‘95 | 1.000 | 1.000 | 1.000 | 1.000 979 800 667 371 500 444 71 
1.00 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .800 | .667 | 37 500 440/421 
! | 
Proportion of emplovees considered satisfactory = .50 
00 | .S00 500 300 | .300 | 500 | 500 500 | .500 .500 500 | 500. 
0s | 1852 | 544 | .338 | 529.324 | 520 | 316 | 512 | 509 | 505 | 303 
10 | .602 588 570 358 | .348 540 532 525 518 510 305 
1S | .653 631 005 2 587 | .373 560 | 349 | 538 526 515 | 50S 
20 | .702 674 640 617 | .598 581 3565 | .550 3535 319 | 3 
25 | .748 | .716 | 676 | 647 | .623 | .602 | .582 | .563 | 544 | 524 | 3:3 
30 | .793 157 712 | .678 649 623 599 576 353 529 515 
35 | 834 797 747 .709 676 646 617 590 562 533 518 
40 | .872 835 783 741 703 669 636 603 571 537 520 
45 | 905 871 818 aa 732 693 655 617 580 341 521 
50 | .933 903} .853 807 762 718 675 631 588 S45 523 
55 | .956 .932 886 840 793 745 696 646 597 548 524 
00 | .974 956 918 875 826 173 718 660 604 3351 $25 
65 | .986 975 946 908 361 804 740 675 612 353 326 
70 | .994 988 969 940 896 836 764 689 617 554 | 526 
75 | 998 .995 986 968 932 872 788 .700 622 335 | .526 
.80 | 1.000 999 996 988 966 909 810 .709 624 3355 226 
85 | 1.000 | 1.000 999 .998 990 | .947 827 113 625 356 2.6 
90 | 1.000 | 1.000 | 1.000 | 1.000 999 | .980 833 714 625 556 a: 
95 1.000 | 1.000 | 1.000 | 1.000 | 1.000 999 833 714 623 336 326 
1.00 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .833 | .714 | .625 | 336 526 
Proportion of employees considered satisfactory = .60 
00 | .600 600 600 600 600 .600 600 600 .600 .600 600 
05} .650 642 634 628 624 620 616 612 609 605 603 
10 | .697 684 667 656 647 639 632 625 618 610 606 
1S | .742 723 .700 684 671 659 648 637 .626 615 608 
20 | .784 761 733 712 695 679 664 .650 636 620 611 
25 | 824 .798 764 .740 719 .700 681 663 645 625 614 
30 | .859 832 796 768 743 721 699 677 655 630 617 
35 891 | 864 826 796 .768 ue 717 691 664 | .636 619 
40| 918 | 893 | .855 | 823 | .793 | 27 64 735 | .706 | .675 | 641 | .622 
45 942 .920 883 851 819 7 88 | 755 | .721 685 646 .624 
ee pri ee eae Ee ee ee a ee ee ee ee 
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TABLE 1—(Continued) 
et 
Selection ratio 





fob 





.050 | .100 | .200 300 | .500 | .600 | .700 | .800 | .900 | 


Proportion of employees considered satisfactory = .60 





Gn. 961 | eoe2 e910" | 8781846 | B12 | 776 | «737 | 695 | 650 | .626 
EsemeOToueecsimiee cai | 605.1 872 | 837 | .7o7 | .754 | .706 | 65s | .628 
60 | 986 | .976 | .985 | .930 | 900 | .863 | .820 | .771 | .716 | .659 | .629 
6s | .993 | .987 | .972 | 953 | .926 | .g91 | .845 | .789 | .727 | .662 | .630 
70 | .997 | .994 | 986 | .972 | 951 | .919 | 872 | 808 | .736 | 664 | 631 
75 | 999 | .998 | .994 | .987 | .973 | .948 | .900 | .826 | .743 | 666 | .631 
30 | 1.000 | 1.000 | .998 | .996 | .990 | .974 | 1930 | .843 | .748 | 666 | .632 
35. 1.000 | 1.000 | 1.000 999 | 998 992 | 960 | 854 | .750 | 667 | 632 
90 | 1,000 | 1.000 | 1.000 | 1.000 | .986 | 887 | .750 | 

: 000 | 1.000 1.000 | 1.000 1.000 | 999 857 jhe .750 ||) 667 f} .632 

1.00 | 1.000 | 1,000 | 1.000 1.000 | 1.000 | 1.000 | 1.000 | .857 | 750 | 





| 667 | .632 
Proportion of employees considered satisfactory = .70 

00 | .700 .700 700 700 700 oo | .700 .700 .700 700 | .700 
0S | .746 739 732 726 722 an 715 me 708 705 | .703 
Om e787 776 762 752 Tht 737 .730 723 ala 709 | .708 
AS 826 810 791 aan 766 755 745 | .736 J25n) 714" 708 
20 | .860 842 819 802 787 Gat 761 748 735 Pomel 
25 | .890 Bal 846 826 809 793 aaa 761 744 | 725 i .71¢ 
30 | .916 897 871 849 830 812 793 | 774 754 Wail 718 
35 | .938 920 894 872 852 ail BiOm) E7880 .76% | Ga Wel 
743 724 











| 

40 | .956 O41 916 894 873 851 .828 803 ) 775 | 
45| 970 | 958 | 936 | 915 | .go4 | 871 | 846 | 818 | .786 | .749 | 727 
50 | .981 972 | 953 935 | 914 891 | 865 32% 198 Ngo | 730 
55 | .989 | 982 | .968 | 952 | 934 | 912 | 885 | .851 | 811 | .761 | .732 
60} 994 | 990 | 980 | 968 | .952 | 932 | .905 ; 870 | 823 | .766 | .734 
65 | .997 | .995 | 989 | .981 | 969 | .952 | .926 | 889 ; 836 | .771 | .735 
70| 999 | .998 | 995 | 990 | 982 | 970 | 948 | 909 | 849 | .774 | .736 
75 | 1.000 | 1.000 | 1.000 | .996 | .992 | 984 | 968 | .931 | 861 TS || ces 
80 | 1.000 | 1.000 | 1.000 | .999 | .998 | .994 | .986 | .953 | .870 | .778 | .737 
85 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .999 | .997 | .975 | 874 | .778 | .737 
.90 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |  .992 875 | .778 | .737 
‘95 | 1,000 | 1.000 | 1.000 | 1.000 | 1.000 ; 1.000 | 1.000 | .992 | 873 | .778 | .737 
1.00 | 1.000 | 1.000 | 1.000 | 1.000 | 1,000 | 1.000 | 1.000 | 1.000 | .875 | 778 | .737 

i | 
Proportion of employees considered satisfactory = .80 

00 800 | .800 | .800 | .800 | .800 | .800 | .800 | .800 | .800 | .800 | .800 
0S 839 | 833 827 823 | .819 | .816 | .813 | .810 | .807 | .804 | .802 
10 872 | 864 | .852 844 | 838 832)1} .826 | .821 | .815 | .809 | .80S 
1S | 901 890 | .876 | .865 | .856 | .848 | 840 | .832 | .823 | 813 | .808 
20 | .926 | 914 | .898 | .885 | .874 | 864 | 853 | .843 | .832 | .819 | 811 


| 
251 946 | .934 | 917 | .904 | .g91 | .880 | .867 | .855 | Sat | 824 | 81 


50H 7.962 951 935 921 .908 895 882 .867 .850 830 818 
3D 974 965 950 937 924 911 896 879 .860 836 821 
40 983 .976 .964 952 .939 926 910 892 871 843 ‘629 
45 .990 984 975 964 953 940 925 906 882 850 829 
50 .994 991 984 975 .966 .954 939 .920 894 857 .832 
‘5 997 995 990 984 977 967 953 934 | .907 865 836 

920 872 838 


60 999 998 995 991 985 .978 .966 949 
65 | 1.000 1.000 1.000 1.000 .992 987 978 963 
70 | 1.000 1.000 1.000 1.000 1.000 994 .988 977 
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TABLE 1—(Contsnued) 
a A 
Selection ratio 
P| 
oso | 100 | 200 | 300 | 400 | 500 | 00 | 700 | 800 | 900 | 950 


Proportion of employees considered satisfactory = .80 
a 


15 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .995 | 988 | .963 | .8a7 | gap 
80 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .999 | 996 | 977 | .888 | 94) 
85 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .999 | 990 | 989 | 449 


95 | 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 .998 .889 .842 


| 
‘90 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 3.000 | 1.000 | 1.000 | .908 | ago | 842 
1.00 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 889 | 842 





Proportion of employees considered satisfactory = .90 








00 | .900 | .900 | .900 | .900 | 900 | 900 | .900 | 900 ; .900 | .900 | 900 
os | 928 | 924 | 920 | 917 | 914 | 912 | .910 | 908 | .906 | .903 | .902 
10 | 949 | 944 | .937 | .932 | 928 | 924 | .920 | 916 | 912 | .907 | 904 
15 | 966 | .960 952 | 946 | 940 | 935 | .930 | 924 | 918 | 911 | 907 
20 | .978 | .972 | .965 | .958 | 952 | .946 | .940 | .933 | .923 | 916 | 9:0 
25! .986 | 982 | .975 | 969 | 963 | 957 | 950 | .942 | 933 | 921 | 913 
30 ©.992 | 989 | .983 | .978 | .972 | 966 | .959 | .951 | .941 | .927 | 91; 
35} 996 | .993 | .989 | .985 | 980 | .975 | 968 | 960 | .949 | .933 | 921 
40 | 998 | .996 | .994 | .990 | 987 | 1982 | 976 | 1968 | 958 | .940 | ‘926 
45 999 | 998 | .996 | 994 | 902 988 | 983 | .976 | .966 | 947 | 930 
50 1,000 | 1.000 | 1.000 | 1.000 | 995 | 993 | 989 | 984 | 974 | 955 | .935 
35 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 994 | .990 | .982 | .962 | 939 
60 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | .994 | .989 970 | .943 
65 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 , 1.000 | 1.000 | 1.000 | .994 978 946 
70 1,000 | 1.000 | 1.000 | 1.000 ' 1.000 | 1.000 | 1.000 | 1.000 | .998 | 985 ; 943 
73. 1.000 | 1.000 | 1.000 | 1.000 1.000 | 1.000 _ 1.000 1.000) 999 | 991 | 947 
80 | 1.000 | 1.000 | 1.000 | 1.000 {| 1.000 | 1.000 | 1.000 | 1.000 1.000 | .996 v4; 
83 | 1.000 | 1.000 1.000 ; 1.000 | 1.000 | 1.000 | 1.000 ; 1.000 ; 1.000 | 999 | vai 
90 | 1.000 | 1.000 , 1.000 | 1.000 | 1.000 | 1.000 | 1.000 ; 1.000 | 1.000 , 1.000 | 247 
95 | 1.000 | 1.000 | 1.000 1.000 , 1.000 | 1.000 | 1.000 | 1.000 , 1.000 | 1.000 | .¥4i 
1.00 | 1.000 | 1.000 | 1.000 § 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 1.000 | 947 
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